Electronic catalog servers, such as those associated with large e-commerce sites, often store large quantities of catalog data to which updates are frequently made. The updates may be received over an extended period of time, such as several months or years, and the quantity of the updates over time may be sufficiently large to require the use of archive storage systems in addition to a live data store. Users and administrators of electronic catalog systems may seek to access the updates made to the catalog data, e.g., for various troubleshooting or marketing purposes. In order to facilitate access to the updates, there may be live or dynamic indexing performed on the live data store. However, such dynamic indexing may affect the throughput and performance of the electronic catalog systems, and may also require increasingly larger storage space for the dynamic index over time.
Archive storage systems may work well for storing hundreds or thousands of objects that are on the order of multiple megabytes. However, such storage systems often do not work as well for storing hundreds of millions to billions of very small records (e.g., kilobytes in size). Additionally, when querying archived data, retrieving the desired archived data is often a challenge that requires a separate search of the archive media itself.
Accordingly, it is desirable to have techniques for querying archived data sets in an efficient manner.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning “having the potential to”), rather than the mandatory sense (i.e., meaning “must”). Similarly, the words “include,” “including,” and “includes” mean “including, but not limited to.”